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Abstract 

Otago Polytechnic (OP) is currently redesigning and redeveloping its programmes and 
courses under an institution-wide initiative called Designing for Learner Success (D4LS). 

This initiative has several phases—design, development, delivery, and evaluation. This 
article focuses on the processes and considerations in the development phase of D4LS, 
which adopted a collaborative and agile approach to developing high-quality course content, 
learning activities, assessments, and resources. A thematic analysis was conducted from a 
record of challenges and limitations encountered by e-learning designers in the development 
process. Four themes emerged as the key challenges: workload and timelines, products and 
process, communication, and capabilities and attitudes. Recommendations are offered for 
improving the support for e-learning designers and course developers who are undertaking 
similar institutional initiatives. 
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Introduction 

Otago Polytechnic (OP) is currently redesigning and redeveloping its programmes and courses 
under an institution-wide initiative called Designing for Learner Success (D4LS). This initiative 
was established in 2015 with the aims of improving overall learner success rates, enhancing 
learner satisfaction with their learning experience, and improving the integration of development 
of learner capabilities within programmes at OP (Otago Polytechnic, 2014). This initiative is in 
line with the Tertiary Education Strategy developed by New Zealand’s Ministry of Education, 
which emphasises the importance of focusing on student achievement while “ensuring tertiary 
education supports development of transferable skills” (Ministry of Education, 2015, para 6). 

The D4LS initiative commenced with programmes arising from the New Zealand Qualifications 
Authority’s (NZQA’s) Targeted Review of Qualifications (TRoQ). It also targeted OP 
programmes that have less than 70% course completion rates, and programmes with large student 
numbers that might have an impact on the whole institution (Otago Polytechnic, 2014). 

The D4LS initiative has four key phases: design, development, delivery, and evaluation, and 
each programme and course has to go through each phase of the D4LS. Otago Polytechnic 
shaped D4LS to take central leadership for programme and course redevelopment and to bring 
together several of OP’s support services—Learning and Teaching, the Quality Enhancement 
Centre, OP Online, and Staff Capability—to work in partnership with the lecturers as subject 
matter experts (SMEs), so ensuring the best use of available expertise and experience. This 
article focuses primarily on the development phase of the D4LS process and explores the 
challenges and limitations encountered by a group of e-learning designers, who also have the role 
of developers (OP Online team) in redesigning and redeveloping courses for three programmes at 
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OP. The collaborative nature of the initiative in general, and the development phase in particular, 
offered great opportunities for course redesign and redevelopment to meet learners’ needs to help 
them succeed—which is the overall aim of the D4LS initiative. The collaborative nature of the 
D4LS initiative also posed some challenges and limitations. As an e-learning designer, I will 
look at the challenges the team of e-leaming designers faced in this collaborative process in the 
development phase, and offer some recommendations. There is a paucity of research on 
collaborative programme and course development approaches (particularly in the context of New 
Zealand tertiary education), and this article goes some way to fill that gap by pointing out the 
avenues to consider when conducting similar initiatives. 

The goals of D4LS, which include improved learner achievement and outcomes, will be 
measured by successful course and qualification completions, increased learner employability, 
enhanced learner capability, and learner satisfaction with their learning experiences in the long 
run. However, measuring the outcomes of the D4LS initiative is not within the scope of this 
article. 

The article commences with a literature review on collaborative programme and course 
development approaches to highlight the lack of substantial studies on institution-wide initiatives 
across New Zealand. This will be followed by an introduction to the four key phases of the D4LS 
initiative, with a focus on the development phase. I will then discuss the challenges encountered 
by e-leaming designers in the development phase and, finally, make some recommendations to 
mitigate the issues discussed. 

Collaborative course development initiatives 

With the advancement of educational technologies, the demand for high-quality online education 
continues to grow. As a result, institutions are faced with the challenge of developing methods 
and processes for online course design and development to most effectively meet their learners’ 
needs and help them succeed. It is evident that developing programmes and courses is a complex 
and multifaceted process that requires careful planning and different levels of expertise. In some 
conventional institutions, course design and development involves only SMEs (Chao, Saj & 
Hamilton, 2010). Although SMEs are experts in their subject areas, they may not have the 
necessary specialised knowledge and capabilities to design and develop courses within the 
overall curriculum and across the programme(s). The consequences of the ever-increasing range 
of online learning and emerging technologies are that designing a high-quality course requires 
other expertise, such as instructional design and designing for e-leaming. As such, “quality 
courseware production requires a highly organized, concerted effort from many players” (Caplan, 
2004, p. 186) and is not “possessed by one person” (Chao et al., 2010, p. 107). 

The relevant literature shows that there has been a handful of collaborative institutional course 
development efforts. One example is the study conducted by Chao et al. (2010) in a Canadian 
university. It examines courses developed by teams that used quality standards. The authors 
discuss how the quality of the course was influenced by the relationship between faculty staff 
and instructional designers, and the issues linked to the use of quality standards as a development 
tool. Their findings indicate that the degree of collaboration was determined by (a) the extent of 
the course development and revision needed, (b) the nature of the rapport between the faculty 
member and the designer, and (c) the faculty member’s experience. 

Puzziferro and Shelton’s (2008) study discusses collaborative course development project, at 
Colorado State University-Global campus. They focus on using an integrated and collaborative 
model based on the instructional design theory of active mastery learning. The authors describe 
how team members (such as faculty course developers, instmctional technologists, course 
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technicians, and copyright clearance coordinators) collaborate in different phases to establish 
their institutional model for course development to enhance effective practice. 

Similarly, Hixon (2008) focuses on a particular online programme development and observes 
four collaborative teams that include (a) an instructional designer, (b) an instructional 
technologist, (c) an information resource consultant, (d) a digital media services consultant, and 
(e) a copyright consultant. The teams were to complete the course development in 67 days. The 
findings of the study highlight the importance of having a flexible course development process, 
faculty ownership of the courses, and clear and effective communication among team members 
at the university. 

The related literature also focuses on the design and development of specific online courses or 
aspects related to online course design. Examples include discussion of design principles (Crews, 
Wilkinson, & Neill, 2015), course design guides and strategies (Stavredes & Herder, 2014; Vai 
& Sosulski, 2011), learner satisfaction (Dziuban, Moskal, & Hartman, 2010), course design and 
learner engagement (Gedera, 2014), and courses with learner-centred, experiential activities and 
learning opportunities (Caulfield, 2011; Holtslander, Racine, Furniss, Buries, & Turner, 2012). 

The preliminary inquiry I carried out in searching for information on programme and course 
development approaches revealed a paucity of studies on collaborative approaches to programme 
and course redesign and redevelopment, particularly in the context of New Zealand tertiary 
education. Furthermore, the existing literature does not include e-learning designers’ perspectives 
on course development processes. These perspectives are important, because e-leaming designers 
have the opportunity to think about the factors involved in the design of the course, how that 
thinking can be transformed into the actual design of the course, and how design can enhance 
students’ engagement and interest in the content and activities. This consideration was 
particularly important in OP’s curriculum framework within the D4LS initiative (which may not 
be the case in some institutions). As Caplan (2004) accentuates, it is best to involve the 
e-leaming designers in the course development process from the beginning to ensure the content 
design is based on sound pedagogy and learning theories. E-learning designers also have the 
expertise to select appropriate media for courses and make sure the courses meet the institution’s 
expectations and quality standards. 

The purpose of this article is to describe how a collaborative and agile approach facilitated the 
design and development of high-quality course content. It further addresses the challenges and 
limitations encountered in the development phase. 

The development phase of Designing for Learner Success 

Designing for Learner Success comprises several key phases including (a) design, (b) 
development, (c) delivery and (d) evaluation. It is planned to have all of OP’s programmes and 
courses go through the D4LS process. The duration of the design and development of each 
programme varies depending on the number of courses offered by that programme. In 
preparation for the D4LS process, lecturers are defined as subject matter experts (SMEs). They 
gather information about their programmes and learners to ensure that all relevant issues can be 
addressed in the redesign and redevelopment process. 

The design phase 

During the design phase the SMEs attend several workshops which are facilitated by the D4LS 
facilitation team (members of the Learning and Teaching team). The workshops focus on the 
programme as well as individual courses. In redesigning the programme and courses, several 
essential factors need to be considered, including learner capabilities and skills, prior 
experiences, and other specific aspects. The SMEs examine and redesign the learning activities 
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and assessments that build learner capability for the course learning outcomes in the programme. 
The details of the course content—including topics and modules, activities and assessments, and 
the sequence in which these occur—are then compiled by the SMEs in a document called a 
'blueprint’. While the SMEs work on the blueprint, the Learning and Teaching facilitators and 
the OP Online team examine the blueprint of each redesigned programme and its respective 
courses to ensure that the course learning outcomes are aligned with the graduate profile 
outcomes, face-to-face and online activities, and assessments. 

The development phase 

In the development phase the blueprint of the designed programmes and courses is handed over 
to the OP Online team (e-leaming designers), and the SMEs collaborate with the OP Online team 
to develop the programme’s Moodle courses. OP Online uses an ‘agile methodology’ and creates 
standards, templates, and procedures to oversee the development. The agile methodology or 
‘agile process management’, which was first used in software development, is an iterative and 
flexible method of managing the design and development of products and processes (Schwaber 
& Sutherland, 2014). There are different types of agile methodologies, the most popular of which 
is Scrum. Scrum is simple and flexible, and breaks large projects into small and manageable 
chunks called sprints. The project’s progress is via a series of sprints (APMG International, 

2011). One sprint can be one week long and the sprint goals can include developing online 
activities for one or two courses. In scrum methodology, a project team generally includes SMEs, 
a lead developer called a scrum master, and other developers who are mainly e-learning 
designers. The project team has daily scrum meetings within a sprint period to check the team’s 
progress and resolve any impediments. Based on the scrum methodology, each member has to 
answer three specific questions at daily scrum meetings: 

1. What did I do yesterday that helped the development team meet the sprint goal? 

2. What will I do today to help the development team meet the sprint goal? 

3. Do I see any impediment that prevents me or the development team from meeting the 
sprint goal? 

At the end of each sprint, developers receive feedback from the SMEs on the developed courses, 
and the required action can roll over to the next sprint. Once the SMEs have provided high- 
quality, robust development information to OP Online, the OP Online team uses a series of 
sprints to design and develop activities and assessments as well as the Moodle course interface. 

The redevelopment of the Moodle courses and their features was based on the specifics of 
learning design thinking which, in turn, were based on constructivist theories to create learner- 
centred courses that meet the OP principles of learning design and quality standards. The quality 
standards include learning design standards, course information standards, writing standards, and 
pedagogy standards, which OP has adopted from TANZ eCampus. 1 The redeveloped Moodle 
courses have several standards: 


• The courses are easy to navigate. 

• The important information about the course (including the timetable and assessments) is 
easy to locate. 

• The activity-led topics are arranged into modules or weeks. 

• There are clear instructions for face-to-face and online activities. 

• Learner support resources and guidelines are provided. 


1 TANZ eCampus is a flexible online learning service provided by a group of major tertiary education providers in New 
Zealand, including Otago Polytechnic. 
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• There are multiple opportunities for students to provide feedback on the course content 
and the learning design. 

As specified in Otago Polytechnic Strategic directions 2015-2017 (Otago Polytechnic, 2015), to 
strengthen the quality and learner outcomes of the programmes and courses, the Learning and 
Teaching Strategic Framework and other frameworks (such as Learner Capability, Sustainability, 
and the Maori Strategic Framework) are incorporated in the learning design. According to the 
Learning and Teaching Strategic Framework, all on-campus courses will be taught with a 
blended delivery model that includes online, face-to-face, student-managed, and work-based 
delivery mechanisms. By incorporating cost-effective and flexible modes of delivery, more 
learners in the region can be reached. The new programmes also incorporate experiential learning 
opportunities that include internships, clinical placements, work experience, and engagement in 
work-related knowledge transfer activities. Otago Polytechnic anticipates that, by integrating 
experiential and reflective learning opportunities, OP graduates will have greater confidence that 
they are ready for work. 

All programmes are designed to improve learner capabilities so that OP graduates will have the 
relevant industry knowledge, technical skills, and personal attributes that they need to succeed. 
Furthermore, the integration of the Sustainability Framework with course content and learner 
activities is expected to develop graduates’ skills and knowledge to be sustainable practitioners. 

In terms of the Maori Strategic Framework, relevant Maori knowledge and perspectives are 
integrated purposefully in the programmes and courses where appropriate. For instance, students 
may gain iwi knowledge of significant local places through learning activities, specific readings, 
site visits, and storytelling. 

Within the D4LS goals, OP’s emphasis evolved to offer activity-led, learner-centred, experiential 
learning opportunities to enhance learner experience and capabilities. To design and develop 
interactive and engaging learning activities and assessments, OP Online uses educational 
technologies and software such as Articulate Storyline, Adobe Captivate, Pathbrite, Adobe 
Connect, Moodle, Zaption, and Google Apps. 

When OP Online has completed the course development, two developers rate the redeveloped 
course using OP quality standards to confirm that it is of the highest possible quality. Then the 
redeveloped course is reviewed and signed off by the lead developer and the SME. 

The delivery phase 

In the delivery phase, the redeveloped programmes and courses are offered to the learners. 

During the first semester, there is a clearly designed support system available for the SMEs. If 
the SMEs need any assistance with their new courses, they can fill in an online form with their 
requests. The OP Online team manages the support process accordingly. The requests can relate 
to aspects of teaching, technology, content, and the like (e.g., to make changes to a learning 
activity developed by e-leaming designers, or to add an extra activity to a module). Learners can 
also add feedback on the design and the content of the course—these comments are taken into 
account for the next iteration. Furthermore, an ongoing upskilling process allows staff members 
to either complete online self-directed courses or request face-to-face training sessions to assist 
them to deliver the new courses using engaging educational technologies. 

The evaluation phase 

An evaluation is conducted after the new courses are implemented. Seven redeveloped courses 
were delivered in the first semester of 2016 for the first time, and an evaluation of these courses 
will take place in the near future. As part of the evaluation, learners and lecturers will provide 
feedback on their experiences of learning and teaching with the redesigned and redeveloped 
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courses. In 2016, another 43 programmes are being redesigned and redeveloped under the 
initiative. 

Figure 1 demonstrates the key phases of the redesign and redevelopment process. 
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Figure 1 Key phases of the D4LS initiative 

Challenges encountered in the development phase 

As with any other institution-wide initiatives, D4LS has faced challenges in some of these 
phases. In such institution-wide initiatives it’s not unusual to have tensions between management 
and those who are implementing the project. In the case of D4LS, e-learning designers observed 
no significant tensions that could disrupt or delay the development phase of the three 
programmes. The initiative was sponsored and supported by OP management. In fact, the 
polytechnic invited an external learning design consultant, who was experienced in instructional 
design and curriculum development, to lead the initiative to ensure the process was based on 
OP’s goals and aspirations. The challenges encountered resulted mainly from the size and the 
nature of the initiative, and because the work involved was larger than anticipated. The section 
below focuses on the challenges encountered in the development phase with which I was 
involved as an e-leaming designer. 

Because the e-leaming designers are working on several programmes concurrently, a record of 
risks, challenges, and limitations encountered in the development phase was compiled and 
recorded in a document called D4LS Risks, Assumptions Register. I analysed and categorised this 
document thematically (Mutch, 2005) (and added my own observational journal entries from 
working as an e-learning designer in the team) to identify the key risks and challenges faced by 
the onl in e developers during the development phase. Table 1 summarises these challenges: 
workload and timelines, products and processes, communication, and capabilities and attitudes. 
Some of these challenges (such as increased workload and time constraints) may appear in the 
collaborative process of any online course production (Xu & Morris, 2007). When redesigning 
and redeveloping a curriculum as an institution-wide process (such as with D4LS), the nature of 
such common challenges complicates and affects the continuity and timeline of the phases in the 
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overall project. Despite the challenges I observed that in the wrap-up meetings, which were 
conducted at the end of each sprint for different programmes, the SMEs were pleased with their 
redeveloped courses. 


Table 1 Summary of challenges encountered in the development phase 


Workload and 

timelines 

Products and process 

Communication 

Capabilities and attitude 

Many programmes are 
concurrently going 
through D4LS 

• Understaffed 

• High workload for 
the developers 

• Inadequate 
development 
time 

SMEs’ expectations of 
the products (activities 
and assessments) in a 
short time frame (e.g., 
expecting a large 
number of online 
activities in blended 
learning courses) 

Development of high- 
quality programmes and 
courses in a short time 
frame (some 
programmes spend a 
long time in the design 
phase) 

Collaborating on 

Google Drive with 
multiple teams across 
the institution 

SMEs’ attitudes towards 
change in general (e.g., 
apprehension about losing 
ownership of the 
programme) 

Developer 
unavailability (e.g., 
getting sick or going 
on leave) 

Programmes developed 
for multiple (sometimes 
three) platforms 

Working with staff 
from different 
branches of OP 

Attitude towards 
technology (e.g., some 
programme SMEs’ 
resistance to use Google 
Drive for collaboration) 

SMEs’ unavailability 
during sprints due to 
other commitments 
or during the 
semester break 

Incomplete documents 
(e.g., incomplete 
activity development 
forms) 

Things can get lost in 
translation (from SMEs 
to the lead developer 
and then to other 
developers) 

SMEs’ capability with 
learning design and 
instructional design 

SMEs’ delayed 
responses during 
sprint 

SMEs wanting to reuse 
existing materials 
(developers spending a 
lot of time for quality 
check and Creative 
Commons license) 

Different developers 
working on different 
sprints of the same 
programme 

Bringing new developers 
up to speed (which 
involves the e-learning 
team setting up relevant 
training sessions) 


Workload and timelines 

When the D4LS process was first initiated, the OP Online team comprised only five members. 

As more programmes began to go through D4LS, e-leaming designers’ workloads increased. 
They were working closely with the SMEs on course design, preparing for the development 
phase (creating templates, support documents, and researching appropriate tools and resources), 
and supporting the academic staff with day-to-day issues and enquiries about educational 
technologies. As a result, the time allocated for the development of courses was very focused and 
limited. To manage the course development process efficiently, the sprints were generally well 
structured and required maximum commitment from the members involved. However, some 
common challenges faced during the sprints were: developers getting sick or going on planned 
leave, and SMEs’ unavailability or delayed responses during the sprints as a result of their 
teaching and other commitments (such as conferences). Some of these challenges are 
unavoidable consequences of the natural course of academic life and work. In spite of their time 
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constraints in terms of their teaching load, administrative roles, and other commitments, SMEs 
were generally positive and supportive of the development initiative. 

Products and processes 

Because many programmes were going through D4LS at once, it was challenging for the 
e-leaming designers/developers to create a large number of high-quality products in a short time. 
In some cases, the SMEs spent more time on the design phase and, therefore, by the time the 
courses got to the development phase, there was only a couple of months left to develop a 
number of courses. Some courses also had to be developed for multiple platforms such as 
Moodle, TANZ eCampus, and Open Education Resources Universitas (OERU), a requirement 
that put pressure on the e-leaming designers. 

Before the developers could build activities and assessments, the SMEs needed to provide 
detailed information such as the name of the activity, module and topic names, learning 
outcomes or learning objectives, a brief description of the activity, a set of instructions (what to 
do now, what to do next), relevant links to resources, and so on. The main challenge faced by the 
developers with regard to this process was receiving incomplete activity and assessment 
development forms. In many instances the developers had to wait until they received enough 
details to be able to develop the course content and other components during sprints. In addition, 
when SMEs wanted to reuse some of the existing materials, the developers had to spend a lot of 
time checking their relevance, quality, and permission to use (e.g., images subject to creative 
commons licences). 

Communication 

Because multiple teams were working together to redesign and redevelop courses, Google Drive 
was used as a common workspace. Developers therefore had to guarantee standard file names 
and a clear folder structure in Google Drive so that everyone could collaborate without getting 
confused and (perhaps) frustrated. Although the developers were keeping track of all the changes 
that were being made to files and folders in Google Drive, sometimes people dropped files in the 
wrong place, duplicated files, and deleted files accidentally. Such instances affected and slowed 
down the communication process. 

Otago Polytechnic has campus branches in three different locations—Dunedin, Cromwell, and 
Auckland. Organising sprint and review meetings with staff at different branches was 
challenging at times. Generally, there were three sprints (each sprint was 3 to 5 days) scheduled 
to build and assemble two blended learning courses. In some instances (when an e-leaming 
designer transferred to another department at OP, and when a developer went on leave), different 
developers had to work on different sprints in the same course. The main challenge in this 
situation was that it took a while for the developers to get to know what had been done and what 
needed to be done, and then start building. Also, if a developer was not able to attend a daily 
scrum meeting, the lead developer then had to explain what needed to be done, and some 
essential details could be lost in translation, transferring information, and updates. 

Capabilities and attitudes 

The SMEs generally had a positive attitude towards the D4LS process of centralised leadership 
and support for the redesign and redevelopment of programmes. However, some SMEs were 
resistant to change and some were technologically challenged when it came to working 
collectively in Google Drive. However, it was a learning experience for them to see the benefits 
of Google Drive when several parties were involved in a project. The SMEs also needed support 
to build their capability in learning design and instructional design. In terms of the developers’ 
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capability building, whenever a new developer joined the OP Online team, the members of the 
team were able to conduct relevant training sessions to bring the new developer up to speed. 

To address some of the issues mentioned above, OP proactively hired additional developers for 
the D4LS initiative in 2015 and 2016. In addition, to ease the e-learning designers’ workload, 
some development work has been given to external contractors. When an external contractor 
works with a SME, an OP Online e-leaming designer liaises to ensure the redesign and 
redevelopment processes remain interactive. Furthermore, to support all the programme teams 
efficiently, OP Online developed and introduced a development support site which contains 
resources and links to all the design documents and staff training courses. 

As well as the programme and course design and development workshops, opportunities and 
support have been developed to build SME capability in learning design, learning and teaching, 
assessments, and digital literacy. The Learning Design Toolkit was developed as an interactive 
and dynamic support system for staff to learn and upskill on components and stages of the 
curriculum design process. The Capability Building Suite was developed to provide support on 
course alignment, learning outcomes, assessment, experiential learning, inclusive learning, 
reflective teaching, and learning in groups. Staff could also access support in the form of online, 
self-paced training courses that were built specifically for D4LS, and they could schedule 
individual consultations on teaching approaches and other aspects of pedagogy. Moodle-based 
online support courses include Teaching a Blended Course, Moodle Essentials, Editing a Moodle 
Course, Managing a Programme Page, End of Semester Process, Examples of Resources for 
Blended Courses, Intro to The Blended Template, and Course Editor Agreement. Most of these 
workshops and professional development opportunities are offered by the Staff Capability team, 
the OP Online team, and the Learning and Teaching team. 

Recommendations from an e-learning designer’s perspective 

My first-hand experience as an e-learning designer in the D4LS initiative has given me the 
opportunity to reflect on the challenges encountered during the development process. The 
following recommendations can mitigate some of the challenges described in the previous 
section. 


• Have a back-up e-learning designer/developer to work with the lead developer in each 
programme, so that if the lead developer is unavailable, the back-up developer can take 
over the responsibilities. 

• Have the same developers working on the same programme sprints. 

• Include developers in sprint meetings so that ideas and plans don’t get lost in translation. 

• Agree on SME turnover time during sprints and have a set schedule for sprint meetings. 

• Demonstrate how to use collaborative online spaces (such as Google Drive) effectively 
from the beginning of the project. 

• Demonstrate how to fill in development forms with all the details required for course and 
activity development. 

• Set realistic expectations of the products developed in the given time frame at the 
beginning of the process, and communicate these to the SMEs and other parties involved. 

In spite of the challenges encountered in the development phase of the project, the redeveloped 
courses were developed on time and met the OP quality standards. The e-leaming designers’ 
hope is that the courses will be well received by students and SMEs in the evaluation phase. (At 
the time of writing this article in 2015 and early 2016, the evaluation process was scheduled to 
start in the second semester of 2016.) 
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As tertiary institutions come under increasing pressure to develop quality online learning 
teaching and learning resources in support of effective blended and online learning initiatives, it 
is essential that they have a better understanding of more sustainable approaches to support staff 
and course development efforts. By investigating the opportunities and challenges offered 
through a collaborative and agile approach—including ways to address the challenges in one 
institution in New Zealand—this study has provided insight into this process from an e-leaming 
designer’s perspective. It is hoped that this research will stimulate further research and discussion 
in this important area, particularly by other practitioners and researchers in similar contexts 
and/or undertaking similar institutional initiatives. 
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